1. Field of the Invention
The present invention relates generally to forms processing technology and more particularly to a system and method to create test materials that can evaluate and help improve systems that recognize handprinted fields.
2. Description of Related Art
Forms processing technology can encompass many types of systems and include many steps including capturing hand-printed data from questionnaires and putting the data into a computer. Many organizations doing forms processing use traditional “heads-down” keying from paper (KFP); and “heads-up” keying from image (KFI).
Traditional “heads-down” keying from paper (KFP) is an approach that involves human keyers sitting in front of a computer terminal, and looking down at a form placed on a rack. They read the data placed on the form by the respondent, and manually key this data into the computer using a KFP software package. People are not very consistent when performing routine tasks for long periods of time, and this process is prone to human error. A major source of error with KFP is placing the data in the wrong field.
“Heads-up” keying from image (KFI) is an approach that uses an electronic scanner to scan the forms before sending the electronic image of the form to a computer screen, along with fields where the human keyer is to key in data. The name comes from the fact that the keyer is looking straight ahead at the screen at all times. This method tends to be more accurate since it greatly reduces the incorrect field problem mentioned above for KFP. It is also often faster than KFP. Unfortunately, it still involves humans which are a constant source of errors.
A major problem with capturing handprinted data from forms filled out by human respondents is measuring the accuracy and efficiency of the total system. Former testing methods, such as using a handprint font, may be satisfactory for production readiness tests, but clearly are not adequate to claim to measure handprint recognition accuracy and efficiency. This is partly because they are considered “too neat” and would thus give an artificially high estimate of accuracy relative to the “real world.”
Creating a test deck manually gives realistic variability but is a time-consuming and demanding task, and you only have one unique deck. Creating a corresponding TRUTH file to evaluate the accuracy of the system's processing adds significantly to the complexity of this task. A truth file is an accurate representation of the handprinted data on the forms. To be able to produce a deck of handprinted forms and the corresponding TRUTH file in a more timely and efficient manner would provide a valuable quality assurance tool.
Typical data collection forms request hand-printed response (rather than cursive) when clarity is required. However, there is great variability in the handprints prepared by the population in general. In order to perform adequate system testing and shakedown, a sufficient number of examples are required so that realistic variability can be characterized for use during testing and system evaluation.
Automation and cost savings can be realized by incorporating handprint Optical Character Recognition (OCR), so that computer systems attempt to recognize the handprinted fields, and send low confidence fields to KFI. The method of this patent creates test materials that make it possible to assess the accuracy of an entire form processing system more easily and consistently, as well as measure efficiency, regardless of whether KFP, KFI, OCR, or all of these are used in any combination. This invention simplifies the expensive and laborious process associated with the (hand made) test forms wherein the forms are keyed twice from paper, and discrepancies verified and corrected by a third person.